3D Representation Research Articles

Applications of the Clifford torus to material textures

This paper introduces a new 2D representation of the orientation distribution function for an arbitrary material texture. The approach is based on the isometric square torus mapping of the Clifford torus, which allows for points on the unit quaternion hypersphere (each corresponding to a 3D orientation) to be represented in a periodic 2D square map. The combination of three such orthogonal mappings into a single RGB (red–green–blue) image provides a compact periodic representation of any set of orientations. Square torus representations of five different orientation sampling methods are compared and analyzed in terms of the Riesz s energies that quantify the uniformity of the samplings. The effect of crystallographic symmetry on the square torus map is analyzed in terms of the Rodrigues fundamental zones for the rotational symmetry groups. The paper concludes with example representations of important texture components in cubic and hexagonal materials. The new RGB representation provides a convenient and compact way of generating training data for the automated analysis of material textures by means of neural networks.

Hi-GeoMVP: a hierarchical geometry-enhanced deep learning model for drug response prediction.

Personalized cancer treatments require accurate drug response predictions. Existing deep learning methods show promise but higher accuracy is needed to serve the purpose of precision medicine. The prediction accuracy can be improved with not only topology but geometrical information of drugs. A novel deep learning methodology for drug response prediction is presented, named Hi-GeoMVP. It synthesizes hierarchical drug representation with multi-omics data, leveraging graph neural networks and variational autoencoders for detailed drug and cell line representations. Multi-task learning is employed to make better prediction, while both 2D and 3D molecular representations capture comprehensive drug information. Testing on the GDSC dataset confirms Hi-GeoMVP's enhanced performance, surpassing prior state-of-the-art methods by improving the Pearson correlation coefficient from 0.934 to 0.941 and decreasing the root mean square error from 0.969 to 0.931. In the case of blind test, Hi-GeoMVP demonstrated robustness, outperforming the best previous models with a superior Pearson correlation coefficient in the drug-blind test. These results underscore Hi-GeoMVP's capabilities in drug response prediction, implying its potential for precision medicine. The source code is available at https://github.com/matcyr/Hi-GeoMVP. Supplementary data is available at Bioinformatics online.

3D Geometry and Slip Distribution in the Long Beach Earthquake Gate, Newport–Inglewood Fault, Los Angeles, California

ABSTRACT We present a new, 3D representation of the Long Beach restraining bend system along the Newport–Inglewood fault (NIF), Los Angeles, California. The NIF is an active strike-slip system that cuts over 60 km through densely populated metropolitan Los Angeles and poses one of the greatest deterministic seismic hazards in the United States (California Division of Mines and Geology, 1988). Part of the NIF sourced the 1933 M 6.4 Long Beach earthquake, which claimed ∼120 lives and remains one of the deadliest events in California history (Barrows, 1974; Hauksson and Gross, 1991). The event is thought to have arrested at Signal Hill within the Long Beach restraining bend, which is formed by a left step in the NIF (Hough and Graves, 2020). Events that rupture through Signal Hill could generate larger (M ≈7) events that pose a significant hazard to urban Los Angeles. Our analysis integrates a diverse range of datasets, including over 4200 fault and horizon penetrations from 243 wells, 2D seismic reflection surveys, field maps, machine-learning-based tomography studies, and the U.S. Geological Survey QFaults surface traces. We show that the fault system in Long Beach has three main strike-slip segments connected by orthogonal reverse faults. The strike-slip faults are nonvertical and nonplanar, merge at depth, and extend through the seismogenic crust. The Long Beach restraining bend system presents numerous rupture pathways and arrest points that NIF earthquakes may follow. We apply a novel, map-based restoration to quantify how much total slip has passed through each of the fault segments. About 375 m of total slip is partitioned into the three main fault strands, including the Reservoir Hill fault (≈75 m), the Northeast Flank fault (≈120 m), and the Cherry Hill fault (≈209 m). This slip partitioning informs our understanding of the tendency of ruptures to involve different fault segments or arrest at specific junctures.

Characterizing urban spatial structure through built form typologies: A new framework using clustering ensembles

Prior research on urban form typologies has largely relied on qualitative classification methods, resulting in subjective and limited analyses. Recently, the emerging data-intensive studies often use a single clustering algorithm and parameter setting, raising concerns about the reliability of the findings. This paper introduces a novel clustering analytical framework for conducting typological studies on urban form that yield stable and reliable results. We employ clustering ensembles, which can combine multiple clustering algorithms to further provide a comprehensible output that facilitates interpretation and knowledge generation. By applying the new framework using 3D building data in Guangzhou, we identify eight typologies of urban built forms and reveal a consistent polycentric pattern across different clustering algorithms and parameter settings. The findings have implications for urban land use planning and regulations by integrating 3D representations of urban form.

A Color- and Geometric-Feature-Based Approach for Denoising Three-Dimensional Cultural Relic Point Clouds

In the acquisition process of 3D cultural relics, it is common to encounter noise. To facilitate the generation of high-quality 3D models, we propose an approach based on graph signal processing that combines color and geometric features to denoise the point cloud. We divide the 3D point cloud into patches based on self-similarity theory and create an appropriate underlying graph with a Markov property. The features of the vertices in the graph are represented using 3D coordinates, normal vectors, and color. We formulate the point cloud denoising problem as a maximum a posteriori (MAP) estimation problem and use a graph Laplacian regularization (GLR) prior to identifying the most probable noise-free point cloud. In the denoising process, we moderately simplify the 3D point to reduce the running time of the denoising algorithm. The experimental results demonstrate that our proposed approach outperforms five competing methods in both subjective and objective assessments. It requires fewer iterations and exhibits strong robustness, effectively removing noise from the surface of cultural relic point clouds while preserving fine-scale 3D features such as texture and ornamentation. This results in more realistic 3D representations of cultural relics.

Three-Dimensional Columnar Microstructure Representation Using 2D Electron Backscatter Diffraction Data for Additive-Manufactured Haynes®282®.

This study provides a methodology for exploring the microstructural and mechanical properties of the Haynes®282® alloy produced via the Powder Bed Fusion-Electron Beam (PBF-EB) process. Employing 2D Electron Backscatter Diffraction (EBSD) data, we have successfully generated 3D representations of columnar microstructures using the Representative Volume Element (RVE) method. This methodology allowed for the validation of elastic properties through Crystal Elasticity Finite Element (CEFE) computational homogenization, revealing critical insights into the material behavior. This study highlights the importance of accurately representing the grain morphology and crystallographic texture of the material. Our findings demonstrate that created virtual models can predict directional elastic properties with a high level of accuracy, showing a maximum error of only ~5% compared to the experimental results. This precision underscores the potential of our approach for predictive modeling in Additive Manufacturing (AM), specifically for materials with complex, non-homogeneous microstructures. It can be concluded that the results uncover the intricate link between microstructural features and mechanical properties, underscoring both the challenges encountered and the critical need for the accurate representation of grain data, as well as the significance of achieving a balance in EBSD area selection, including the presence of anomalies in strongly textured microstructures.

Accelerated semantic segmentation of additively manufactured metal matrix composites: Generating datasets, evaluating convolutional and transformer models, and developing the MicroSegQ+ Tool

Data-driven applications are penetrating every aspect of Additive Manufacturing (AM) to enable efficient resolution of key existing challenges. One such application is semantic segmentation which automatically quantifies post-process structure data from graphic or 3D computer representations. In industrial settings, these models can expedite the development of new materials or processes. However, Additively Manufactured (AMed) materials offer unique challenges, which require new models to be developed and trained. These challenges include out-of-balance classes (e.g., defects), massive labeling efforts, and significant data preparation costs. Recent applications of semantic segmentation in AM have shown the potential of ensemble-based approaches at the expense of increased parameters and computational burden while ignoring the minority classes. In this work, we propose a reproducible method and develop an associated tool to rapidly segment and subsequently quantify industrial AMed metallographic images. First, two new datasets representing AMed materials are generated from extensive experimentation. Subsequently, state-of-the-art models from convolutional and self-attention categories are evaluated in their ability to segment the datasets of interest. Finally, a modular software tool is developed for industrial applications. The developed Microstructure Segmentation, Quantification, and Fusion (+) tool or MicroSegQ+ enables a weighted pairing of predictions from different semantic segmentation models. The pairing strategy helps to exploit the complimentary performance of convolutional and transformer models on multi-class metallographic images from a Metal Matrix Composite (MMC) material system. In addition to developing models on this MMC dataset, the proposed strategy is evaluated on an open-source metal AM dataset. We also evaluate the generalizability of the best-performing model on a second MMC dataset from a different industrial setting that shows an overall accuracy of 93 % without fine-tuning. The proposed tool enables rapid quantification under 1 min, unlike the existing semi-automatic approach that takes 1–2 h of segmentation effort on single bead cross-sections. The tool is wrapped into a single executable file for industrial deployment. The work also contributes to two annotated datasets from a direct energy deposition (DED) AM process. The datasets can be used to develop and validate new semantic segmentation models of AMed microstructures, whereas the functionality of the tool can be expanded upstream to include data and modeling steps, enabling a no-code pipeline for the industry.

BrutNet: A novel approach for violence detection and classification using DCNN with GRU

AbstractAutomatic Violence Detection and Classification (AVDC) with deep learning has garnered significant attention in computer vision research. This paper presents a novel approach for combining a custom Deep Convolutional Neural Network (DCNN) with a Gated Recurrent Unit (GRU) in developing a new AVDC model called BrutNet. Specifically, a time‐distributed DCNN (TD‐DCNN) is developed to generate a compact 2D representation with 512 spatial features per frame from a set of equally‐spaced frames of dimension 16090 in short video segments. Further to leverage the temporal information, a GRU layer is utilised, generating a condensed 1D vector that enables binary classification of violent or non‐violent content through multiple dense layers. Overfitting is addressed by incorporating dropout layers with a rate of 0.5, while the hidden and output layers employ rectified linear unit (ReLU) and sigmoid activations, respectively. The model is trained on the NVIDIA Tesla K80 GPU through Google Colab, demonstrating superior performance compared to existing models across various video datasets, including hockey fights, movie fights, AVD, and RWF‐2000. Notably, the model stands out by requiring only 3.416 million parameters and achieving impressive test accuracies of 97.62%, 100%, 97.22%, and 86.43% on the respective datasets. Thus, BrutNet exhibits the potential to emerge as a highly efficient and robust AVDC model in support of greater public safety, content moderation and censorship, computer‐aided investigations, and law enforcement.

Feasibility of an MR-based digital specimen for tongue cancer resection specimens: a novel approach for margin evaluation.

This study explores the feasibility of ex-vivo high-field magnetic resonance (MR) imaging to create digital a three-dimensional (3D) representations of tongue cancer specimens, referred to as the "MR-based digital specimen" (MR-DS). The aim was to create a method to assist surgeons in identifying and localizing inadequate resection margins during surgery, a critical factor in achieving locoregional control. Fresh resection specimens of nine tongue cancer patients were imaged in a 7 Tesla small-bore MR, using a high-resolution multislice and 3D T2-weighted Turbo Spin Echo. Two independent radiologists (R1 and R2) outlined the tumor and mucosa on the MR-images whereafter the outlines were configured to an MR-DS. A color map was projected on the MR-DS, mapping the inadequate margins according to R1 and R2. We compared the hematoxylin-eosin-based digital specimen (HE-DS), which is a histopathological 3D representation derived from HE stained sections, with its corresponding MR-images. In line with conventional histopathological assessment, all digital specimens were divided into five anatomical regions (anterior, posterior, craniomedial, caudolateral and deep central). Over- and underestimation 95th-percentile Hausdorff-distances were calculated between the radiologist- and histopathologist-determined tumor outlines. The MR-DS' diagnostic accuracy for inadequate margin detection (i.e. sensitivity and specificity) was determined in two ways: with conventional histopathology and HE-DS as reference. Using conventional histopathology as a reference, R1 achieved 77% sensitivity and 50% specificity, while R2 achieved 65% sensitivity and 57% specificity. When referencing to the HE-DS, R1 achieved 94% sensitivity and 61% specificity, while R2 achieved 88% sensitivity and 71% specificity. Range of over- and underestimation 95HD was 0.9mm - 11.8mm and 0.0mm - 5.3mm, respectively. This proof of concept for volumetric assessment of resection margins using MR-DSs, demonstrates promising potential for further development. Overall, sensitivity is higher than specificity for inadequate margin detection, because of the radiologist's tendency to overestimate tumor size.

Data-Driven Car Drag Prediction With Depth and Normal Renderings

Abstract Generative artificial intelligence (AI) models have made significant progress in automating the creation of 3D shapes, which has the potential to transform car design. In engineering design and optimization, evaluating engineering metrics is crucial. To make generative models performance-aware and enable them to create high-performing designs, surrogate modeling of these metrics is necessary. However, the currently used representations of 3D shapes either require extensive computational resources to learn or suffer from significant information loss, which impairs their effectiveness in surrogate modeling. To address this issue, we propose a new 2D representation of 3D shapes. We develop a surrogate drag model based on this representation to verify its effectiveness in predicting 3D car drag. We construct a diverse dataset of 4535 high-quality 3D car meshes labeled by drag coefficients computed from computational fluid dynamics simulations to train our model. Our experiments demonstrate that our model can accurately and efficiently evaluate drag coefficients with an R2 value above 0.84 for various car categories. Our model is implemented using deep neural networks, making it compatible with recent AI image generation tools (such as stable diffusion) and a significant step toward the automatic generation of drag-optimized car designs. Moreover, we demonstrate a case study using the proposed surrogate model to guide a diffusion-based deep generative model for drag-optimized car body synthesis.

3D modeling of the stratigraphic and structural architecture of the Crotone basin (southern Italy) using machine learning with Python

The 3D modeling and representation of geological data have experienced significant growth within last years, due to the use of new technologies derived from advancements in land representation methods. These technologies enable interactive, intuitive and clear geological visualizations. This paper shows how, by using the open-source Python software (operable with a simple internet browser) for machine learning (linear and KNN interpolations), together with Geographic Information Systems (GIS), it is possible to achieve interactive 3D visualizations of geological features in sedimentary basins. This study is performed in the onshore-offshore Crotone area (southern Italy) where a large amount of stratigraphic datasets are available from core perforation and seismic profiles due to the presence of a natural gas extraction field. Thanks to a database of 63 drilling lithologic records and 43 check points obtained from 9 interpreted seismic sections, several 3D HTML models were constructed defining three stratigraphic units (Pre-Messinian, Messinian, and Post-Messinian). An overlap of the Post-Messinian top surface and an erosional truncation of the Messinian top surface toward the N were observed, together with a rising of the Pre-Messinian top surface in the northwestern area. This stratigraphic architecture may indicate differential subsidence and/or uplifting due to syn-sedimentary fault kinematics in the whole studied area. The 3D models with the stratigraphic unit boundary surfaces obtained with KNN interpolation (showing stepped and abrupt edges) allowed the interpretation in terms of structural architecture and syn-sedimentary fault kinematics. Three main sets of faults were deduced: N–S; NNW-SSE, and ENE-WSW. A minorly represented E-W set was added to the main sets. These faults generated a horsts-grabens structure, and in many cases a determinate set of faults caused a progressive lowering or rising of some areas with an “en echelon” arrangement. According to previous works, these deduced sets of faults (most of them strike-slip faults) have a good agreement with the general structural architecture and defined faults in the area.

Sensorless volumetric reconstruction of fetal brain freehand ultrasound scans with deep implicit representation

Three-dimensional (3D) ultrasound imaging has contributed to our understanding of fetal developmental processes by providing rich contextual information of the inherently 3D anatomies. However, its use is limited in clinical settings, due to the high purchasing costs and limited diagnostic practicality. Freehand 2D ultrasound imaging, in contrast, is routinely used in standard obstetric exams, but inherently lacks a 3D representation of the anatomies, which limits its potential for more advanced assessment. Such full representations are challenging to recover even with external tracking devices due to internal fetal movement which is independent from the operator-led trajectory of the probe. Capitalizing on the flexibility offered by freehand 2D ultrasound acquisition, we propose ImplicitVol to reconstruct 3D volumes from non-sensor-tracked 2D ultrasound sweeps. Conventionally, reconstructions are performed on a discrete voxel grid. We, however, employ a deep neural network to represent, for the first time, the reconstructed volume as an implicit function. Specifically, ImplicitVol takes a set of 2D images as input, predicts their locations in 3D space, jointly refines the inferred locations, and learns a full volumetric reconstruction. When testing natively-acquired and volume-sampled 2D ultrasound video sequences collected from different manufacturers, the 3D volumes reconstructed by ImplicitVol show significantly better visual and semantic quality than the existing interpolation-based reconstruction approaches. The inherent continuity of implicit representation also enables ImplicitVol to reconstruct the volume to arbitrarily high resolutions. As formulated, ImplicitVol has the potential to integrate seamlessly into the clinical workflow, while providing richer information for diagnosis and evaluation of the developing brain.

Diversity of 3D APAR and LAI dynamics in broadleaf and coniferous forests: Implications for the interpretation of remote sensing-based products

Forests substantially mediate the water and carbon dioxide exchanges between terrestrial ecosystems and the atmosphere. The rate of this exchange, including evapotranspiration (ET) and gross primary production (GPP), depends mainly on the underlying vegetation type, health state, and the composition of abiotic environmental drivers. However, the complex 3D structure of forest canopies and the inherent top-view perspective of optical and thermal remote sensing complicate remote sensing-based retrievals of biotic and abiotic factors that eventually determine ET and GPP. This study investigates the sensitivity of remote sensing approaches to 3D variation of abiotic and biotic environmental drivers. We use 3D virtual scenes of two structurally different Swiss forests and the radiative transfer model DART to simulate the 3D distribution of solar irradiance and reflected radiance in the forest canopy. These simulations, in combination with LiDAR data, are used to derive the absorbed photosynthetic active radiation (APAR) and the leaf area index (LAI) in 3D space. The 3D variation of both parameters was quantified and analyzed. We then simulated images of the top-of-canopy bi-directional reflectance factor (BRF) and compared them with the hemispheric-conical reflectance factor (HCRF) data derived from HyPlant airborne imaging spectrometer measurements. The simulated BRF data was used to derive APAR and LAI, and the results were compared to their respective 3D representations. We unravel considerable spatial differences between both representations. We discuss possible reasons for the disagreement, including a potential insensitivity of the inherent top-of-canopy view for the real 3D product dynamics and limitations of the processing of remote sensing data, especially the approximation of effective surface irradiance. Our results can help understanding sources of uncertainties in remote sensing based gas exchange products and defining mitigation strategies.

Towards Compact 3D Representations via Point Feature Enhancement Masked Autoencoders

Learning 3D representation plays a critical role in masked autoencoder (MAE) based pre-training methods for point cloud, including single-modal and cross-modal based MAE. Specifically, although cross-modal MAE methods learn strong 3D representations via the auxiliary of other modal knowledge, they often suffer from heavy computational burdens and heavily rely on massive cross-modal data pairs that are often unavailable, which hinders their applications in practice. Instead, single-modal methods with solely point clouds as input are preferred in real applications due to their simplicity and efficiency. However, such methods easily suffer from limited 3D representations with global random mask input. To learn compact 3D representations, we propose a simple yet effective Point Feature Enhancement Masked Autoencoders (Point-FEMAE), which mainly consists of a global branch and a local branch to capture latent semantic features. Specifically, to learn more compact features, a share-parameter Transformer encoder is introduced to extract point features from the global and local unmasked patches obtained by global random and local block mask strategies, followed by a specific decoder to reconstruct. Meanwhile, to further enhance features in the local branch, we propose a Local Enhancement Module with local patch convolution to perceive fine-grained local context at larger scales. Our method significantly improves the pre-training efficiency compared to cross-modal alternatives, and extensive downstream experiments underscore the state-of-the-art effectiveness, particularly outperforming our baseline (Point-MAE) by 5.16%, 5.00%, and 5.04% in three variants of ScanObjectNN, respectively. Code is available at https://github.com/zyh16143998882/AAAI24-PointFEMAE.

Sparse3D: Distilling Multiview-Consistent Diffusion for Object Reconstruction from Sparse Views

Reconstructing 3D objects from extremely sparse views is a long-standing and challenging problem. While recent techniques employ image diffusion models for generating plausible images at novel viewpoints or for distilling pre-trained diffusion priors into 3D representations using score distillation sampling (SDS), these methods often struggle to simultaneously achieve high-quality, consistent, and detailed results for both novel-view synthesis (NVS) and geometry. In this work, we present Sparse3D, a novel 3D reconstruction method tailored for sparse view inputs. Our approach distills robust priors from a multiview-consistent diffusion model to refine a neural radiance field. Specifically, we employ a controller that harnesses epipolar features from input views, guiding a pre-trained diffusion model, such as Stable Diffusion, to produce novel-view images that maintain 3D consistency with the input. By tapping into 2D priors from powerful image diffusion models, our integrated model consistently delivers high-quality results, even when faced with open-world objects. To address the blurriness introduced by conventional SDS, we introduce the category-score distillation sampling (C-SDS) to enhance detail. We conduct experiments on CO3DV2 which is a multi-view dataset of real-world objects. Both quantitative and qualitative evaluations demonstrate that our approach outperforms previous state-of-the-art works on the metrics regarding NVS and geometry reconstruction.

HDformer: A Higher-Dimensional Transformer for Detecting Diabetes Utilizing Long-Range Vascular Signals

Diabetes mellitus is a global concern, and early detection can prevent serious complications. 50% of those with diabetes live undiagnosed, disproportionately afflicting low-income groups. Non-invasive methods have emerged for timely detection; however, their limited accuracy constrains clinical usage. In this research, we present a novel Higher Dimensional Transformer (HDformer), the first Transformer-based architecture which utilizes long-range photoplethysmography (PPG) to detect diabetes. The long-range PPG maximizes signal contextual information when compared to the less-than 30 second signals commonly used in existing research. To increase the computational efficiency of HDformer’s long-range processing, a new attention module, Time Square Attention (TSA), is invented to achieve linear computational complexity with respect to the token volume while retaining the local/global dependencies. TSA converts the 1D inputs into 2D representations, grouping the adjacent points into a single 2D token. It then generates dynamic patches and feeds them into a gated mixture-of-experts (MoE) network, optimizing the learning on different attention areas. HDformer achieves state-of-the-art results (sensitivity 98.4, accuracy 97.3, specificity 92.8, AUC 0.929) on the standard MIMIC-III dataset, surpassing existing research. Furthermore, we develop an end-to-end solution where a low-cost wearable is prototyped to connect with the HDformer in the Cloud via a mobile app. This scalable, convenient, and affordable approach provides instantaneous detection and continuous monitoring for individuals. It aids doctors in easily screening for diabetes and safeguards underprivileged communities. The enhanced versatility of HDformer allows for efficient processing and learning of long-range signals in general one-dimensional time-series sequences, particularly for all biomedical waveforms.

VLM2Scene: Self-Supervised Image-Text-LiDAR Learning with Foundation Models for Autonomous Driving Scene Understanding

Vision and language foundation models (VLMs) have showcased impressive capabilities in 2D scene understanding. However, their latent potential in elevating the understanding of 3D autonomous driving scenes remains untapped. In this paper, we propose VLM2Scene, which exploits the potential of VLMs to enhance 3D self-supervised representation learning through our proposed image-text-LiDAR contrastive learning strategy. Specifically, in the realm of autonomous driving scenes, the inherent sparsity of LiDAR point clouds poses a notable challenge for point-level contrastive learning methods. This method often grapples with limitations tied to a restricted receptive field and the presence of noisy points. To tackle this challenge, our approach emphasizes region-level learning, leveraging regional masks without semantics derived from the vision foundation model. This approach capitalizes on valuable contextual information to enhance the learning of point cloud representations. First, we introduce Region Caption Prompts to generate fine-grained language descriptions for the corresponding regions, utilizing the language foundation model. These region prompts then facilitate the establishment of positive and negative text-point pairs within the contrastive loss framework. Second, we propose a Region Semantic Concordance Regularization, which involves a semantic-filtered region learning and a region semantic assignment strategy. The former aims to filter the false negative samples based on the semantic distance, and the latter mitigates potential inaccuracies in pixel semantics, thereby enhancing overall semantic consistency. Extensive experiments on representative autonomous driving datasets demonstrate that our self-supervised method significantly outperforms other counterparts. Codes are available at https://github.com/gbliao/VLM2Scene.

LISR: Learning Linear 3D Implicit Surface Representation Using Compactly Supported Radial Basis Functions

Implicit 3D surface reconstruction of an object from its partial and noisy 3D point cloud scan is the classical geometry processing and 3D computer vision problem. In the literature, various 3D shape representations have been developed, differing in memory efficiency and shape retrieval effectiveness, such as volumetric, parametric, and implicit surfaces. Radial basis functions provide memory-efficient parameterization of the implicit surface. However, we show that training a neural network using the mean squared error between the ground-truth implicit surface and the linear basis-based implicit surfaces does not converge to the global solution. In this work, we propose locally supported compact radial basis functions for a linear representation of the implicit surface. This representation enables us to generate 3D shapes with arbitrary topologies at any resolution due to their continuous nature. We then propose a neural network architecture for learning the linear implicit shape representation of the 3D surface of an object. We learn linear implicit shapes within a supervised learning framework using ground truth Signed-Distance Field (SDF) data for guidance. The classical strategies face difficulties in finding linear implicit shapes from a given 3D point cloud due to numerical issues (requires solving inverse of a large matrix) in basis and query point selection. The proposed approach achieves better Chamfer distance and comparable F-score than the state-of-the-art approach on the benchmark dataset. We also show the effectiveness of the proposed approach by using it for the 3D shape completion task.

NeRF-LiDAR: Generating Realistic LiDAR Point Clouds with Neural Radiance Fields

Labelling LiDAR point clouds for training autonomous driving is extremely expensive and difficult. LiDAR simulation aims at generating realistic LiDAR data with labels for training and verifying self-driving algorithms more efficiently. Recently, Neural Radiance Fields (NeRF) have been proposed for novel view synthesis using implicit reconstruction of 3D scenes. Inspired by this, we present NeRF-LIDAR, a novel LiDAR simulation method that leverages real-world information to generate realistic LIDAR point clouds. Different from existing LiDAR simulators, we use real images and point cloud data collected by self-driving cars to learn the 3D scene representation, point cloud generation and label rendering. We verify the effectiveness of our NeRF-LiDAR by training different 3D segmentation models on the generated LiDAR point clouds. It reveals that the trained models are able to achieve similar accuracy when compared with the same model trained on the real LiDAR data. Besides, the generated data is capable of boosting the accuracy through pre-training which helps reduce the requirements of the real labeled data. Code is available at https://github.com/fudan-zvg/NeRF-LiDAR

A Plug-and-Play Quaternion Message-Passing Module for Molecular Conformation Representation

Graph neural networks have been widely used to represent 3D molecules, which capture molecular attributes and geometric information through various message-passing mechanisms. This study proposes a novel quaternion message-passing (QMP) module that can be plugged into many existing 3D molecular representation models and enhance their power for distinguishing molecular conformations. In particular, our QMP module represents the 3D rotations between one chemical bond and its neighbor bonds as a quaternion sequence. Then, it aggregates the rotations by the chained Hamilton product of the quaternions. The real part of the output quaternion is invariant to the global 3D rotations of molecules but sensitive to the local torsions caused by twisting bonds, providing discriminative information for training molecular conformation representation models. In theory, we prove that considering these features enables invariant GNNs to distinguish the conformations caused by bond torsions. We encapsulate the QMP module with acceleration, so combining existing models with the QMP requires merely one-line code and little computational cost. Experiments on various molecular datasets show that plugging our QMP module into existing invariant GNNs leads to consistent and significant improvements in molecular conformation representation and downstream tasks.